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FAMILIARITY WITH SUBORDINATES' JOBS: 
IMMEDIATE VERSUS SECONDARY SUPERVISORS 



K INTRODUCTION 

Supervisor knowledge of a subordinate's job assumes practical importance if one seeks valid estimates 
of job performance. Presumably, the closer a supervisor is to his subordinate, physically and temporally, the 
more he ought to know about his subordinate's job. This means that an "immediate" supervisor should be 
more familiar with the activities of a job incumbent than any other supervisor of that individual. Must one 
always acquire job performance ratings from immediate supervisors, and immediate supervisors only? There 
are organizational and geographical factors which make it nearly impossible to fulfill such a requirement. 
Would this mean that some areas of job performance cannot be satisfactorily investigated? 

The present study compares the knowledge shown by immediate supervisors with that shown by 
other than immediate supervisors concerning tasks perfonncd by their mutual subordinates. A finding that 
there is substantial difference between the two levels of supervisory knowledge of a worker's activities 
could change the design for data collection in soxvt occupational research studies. 

Experience has shown that one can rely on a job incumbent to provide an honest statement of the 
relative time he spends on tasks. Investigations that have been made on job inventory responses indicate 
that they are not only honest but also reasonably accurate. However, very little research has been directed 
toward determining what supervisors know about how subordinates spend their time on the job. Madden, 
Hazel, and Christal reported a study in 1964, in which they compared tasks checked as performed by 
incumbents with corresponding estimates by their supervisors. A 57% agreement was obtained on tasks 
mutually checked, and a 48% agreement was achieved when relative time spent was considered. However, 
that study used only immediate supervisors. 

It should be made clear that the present study is not concerned with the absolute amount of 
knowledge that either supervisor possessed about a subordinate's job. Two levels of supervisor were asked 
to make ratings on the quality of performance of individual tasks. Their instruction was to rate only on the 
tasks they were sure that their subordinates were performing. Thus, supervisors were free to omit ratings on 
tasks which their subordinates could be perfomung, but which the supervisor was not in a position to 
observe. While the Madden et al study involved collection of data from incumbents and supervisors at the 
same time, the present study introduced a lag between incumbent responses and supervisors' ratings that in 
some instances was as great as two months. 

Why should a supervisor fail to rate a subordinate on the performance of a task? Some possible 
reasons are: 

1. The incumbent used to do the task but no longer did it. His job had changed during the interim 
between inventory and rating. 

2. The task was such a small part of the incumbent's job that his supervisor doubted the legitimacy 
of its inclusion. 

3. The supervisor knew more about the meaning of the task statement than the incumbent, and 
knew that the statement called for more than the incumbent performed. For example, an incumbent 
sergeant checked that he reviewed budgets; this could be so if looking for mistakes in arithmetic and 
misspelled words was all that there was to reviewing a budget, but not correct if reviewed meant evaluated. 

4. The supervisor was aware that the incumbent did the task, but considered that his observation 
was insufncient to rate performance. 

5. The supervisor really didn't know that the airman performed the task. 

6. The supervisor knew that the incumbent performed the task but did not feel qualified to judge 
the performance. Only the fifth, of the six listed items, reflects lack of knowledge of the incumbent's 
activities, yet any of them could result in "disagreement." It can be seen from the foregoing that the data 
of this study permit no conclusions about "real" knowledge on the part of the supervisor, but that the 
study still permits comparison of two levels of supervisor in a relative sense, using the incumbents' job 
inventories as the standard. 
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11. PROCEDURE 



Airmen at all skill levels in the Inventory Management, DAFSC 645X0, and Materiel Facilities, 
DAFSC 647X0, career ladders completed a regular job inventory which contained tasks of both ladders in a 
angle booklet. Respondent data were collected in continental United States, Alaska, and three overseas 
bases. Upon return of completed inventories, lists of selected respondents were sent to the bases of origin. 
These airmen were rated by two supervisors, an imn^ediate and an other-than-immediate supervisor who was 
acquainted Nvith the aimian and his job perfomiance. Raters received two sets of rating materials to be 
executed in tlie following sequence: (a) overall performance rating, then ratings on 65 traits; (/?) an 
inventory which required ratings on tasks that the supervisor was sure his subordinate performed and 
ratings on the ability to learn all the tasks in the inventory, including those tasks he was already performing. 
The supervisor was asked to rate each task on the incumbent's comparative perfonnance with other aimien 
who also did the task. For the analyses of this study, the standard for comparison was the set of relative 
time spent ratings made by an incumbent with respect to tasks he performed in his job. 

At this point consider only that an incumbent had shown some of his time spent on a specific task. If 
an immediate supervisor elected to rate an incumbent's performance on that task a tally was given* 7. If 
the other supervisor also rated him on the same task, a double tally was given: 77. If the immediate 
supervisor did not rate the incumbent on the task which llie incumbent had marked, a null was given: 0. If 
the other supervisor had marked the task in question Uie tally would be: 01. The total possible 
combinations are. 77, 01 y 10, and 00. Tally pairs were collected into categories based on time? spent on 
tasks for the purpose of comparing the "agreements" of the two levek of supervisor, which permitted time 
spent gradations. There is also the set of "agreements*' on tasks not performed, that is, on tasks not marked 
by the incumbent. For these tasks there are also four tally combinations of "agreement," but not time 
spent gradations. 

The number of sets of ratings with both an immediate and an other-than-immediate supervisor was 
295, wliich were the only cases retained for the analyses. The number of ratees by skill level is as follows* 
Inventory Management - 64530, 25; 64550, 93; 64570, 58; 64590, 15; Materiel Facilities - 64730, 12; 
64750, 56;64770,31;64790,5. 

Supervisors of the 295 incumbents were a heterogeneous sampling, ranging from "blue collar" 
workers to higher ranking officers and civilians. The scarcity of incumbents in the 64X90 level was due 
both to actual low manning and to the difficulty in obtaining immediate supervisor ratings. The scarcity of 
apprentices, 64X30s, was a normal function of length of time on the job prior to being asked to complete a 
job inventory. Varying amounts of time elapsed between return of tlte incumbent's inventory and his rating 
by supervisors; this was an uncontrollable factor which varied from base to base. 



III. /VNALYSES AND RESULTS 

Tlie standard Comprehensive Occupational Data Analysis Programs (CODAP) (Christal, 1972) convert 
an mcumbent's responses to percent time spent on each task. These percent time spent values were assigned 
to a 15 interval scale, as presented in Table 1 . The number of scale intervals of Table 1 was first established 
on an empirical basis, having been derived from the CODAP data of the study, then converted to an 
arbitrary scale whose intervals followed a systematic progression. Since many incumbents performed the 
same task, and the specific time spent on a particular task was a function of all the tasks an incumbent 
performed, a task might appear in any of the intervals of Table 1. This table is designed to show how the 
1 5-interval scale was generated to accomodate the extreme skewness of the distribution of 
percent-time -spent values arising from the analysis of ordinary job inventory responses. Note from the 
Interval Range and Step columns of Table 1 that the steps become progressively larger. The frequency of 
tasks at a specific interval is given in the column headed Number of Tasks Occurring. A count of the 
different contributors is given in the column Incimbent Contributors Performing the Tasks, This colunm is 
not additive because the total number of different contributors is the number of incumbents in the study, 
295. The 15 interval scale was left with two vacant intervals of very small percent time spent. There are 
some jobs in the Air Force with so many tasks that small fractional percents arc needed, and tlius this scale 
has general applicability for other studies. 



Table I. Frequencies Within tlie 15 Intervals of Percent Time Spent on Tasks 



Intarval 



nttrval Ranf* 
In Perctnts 



Sttp 
SUt 



Numbtr of Tasks 
Occurrlns In tht 
Inttrval 



Numbtr of DIffcrtnt 
Incumbtnt Contributors 
Ptrf orming tht Tasks 



01 
02 
03 
04 
05 
06 
07 
08 
09 
10 
11 
12 
13 
14 
15 



000.001-000.049 
000.050-000.149 
000.150-000.349 
000.350-000.649 
000.650-001.049 
001.050-001.549 
001.550-002.149 
002.150-002.849 
002.850-003.649 
003.650-004.549 
004.550-005.549 
005.550-007.549 
007.550-011.549 
011.550-019.549 
019.550-100.000 



.05 
.10 
.20 
.30 
.40 
.50 
.60 
.70 
.80 
.90 
1.00 
2.00 
4.00 
8.00 



0 
0 
23 
187 

735 
1,001 
1,405 
1,242 
1,012 
510 
383 
561 
399 
227 
100 



0 
0 
6 
38 
66 
81 
117 
124 
133 
110 
85 
117 
98 
68 
39 



Total 



7,785 



295 



Matching supervisor and incumbent responses produced the data given in Table 2. Table 2 provides 
four kinds of agreement between the two levels of supervisor and incumbent. 00, 10, 01 , 1 1. The columns 
designate the kind of agreement, the rows designate the time spent intervals. In addition, there is a totals 
column and an independent column giving the number of airmen who contributed the task data in each 
time spent interval. The agreement categories are mutually exclusive, which makes it possible to convert to 
percentages and to balance the percent sums. The tallies for either level of supervisor are the sum of tallies 
by his level and the tallies by both levels. The data of Table 2 have been broken down into career ladders, 
and then broken down into lower and upper skill level samples. The break-out results have been prepared in 
graphic form, and one of the tables from which the graphs were derived is presented as Table 3. 

The analyses are simple distributions. When presented in graphic form the columns are summed to 
show the agreement on tasks performed for the two levels of supervisor. For example. Table 3, interval 05, 
shows 8.95 agreement under column 10 (other supervisor agrees), and 37.90 under column 11 (both 
supervisors agree), which totals 46.85. That value is found in the interval 05-06 of Figure 1 as the point 
47% on the dashed line, which represents agreement by other than immediate supervisors. Trends which 
cannot be discerned from the tables are easily seen in the graphs. The last two rows of the tables (total tasks 
performed and number of incumbents providing the data) are given in the graphs and can be used to check 
correspondence between table and graph. 

It was observed when half the flgures were completed that successive intervals produced a consistent 
slope when the number of contributors was 20 or more. Instead of collapsing intervals to achieve sufficient 
contributors, vertical lines have been drawn on Figures 1 througli 6 to indicate the span of intervals witliin 
which there were 20 or more contributors. The series of figures successively subdivides the data, first into 
career ladders, then into upper and lower level airmen within ladders. Each figure gives the percent 
agreement between immediate supervisors and incumbents (percent tallies by time spent intervals) as a solid 
Une, corresponding data for other supervisors are dashed lines. There is no theoretical justification for 
connecting these points, but it has been done to aid tlie reader. 

Figures 1 and 2 yield the essential finding of this study. With the aid of the solid and dashed lines it 
can be seen that these cross each other unsystematically, and that tliey follow the same upward progression 
from 40% agreement to 70%. There is no superiority of immediate supervisors over other supervisors in 
Figures 1 and 2. This finding cannot be attributed to the baseline used. In the development of the time 
spent scale data were first analyzed on a "scale" of irregular time intervals composed of nearly equal 
numbers of tasks producing an interval. Graphs similar to those presented here showed the same result as 
these, regardless of baseline. 
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Table 2. Supervisor-incumbent Agreement on Task Performed, Both Ladders 







00 


10 


01 


1 1 




Incumbents 






Ntithtr 


Othtr Sup«r* 


lmm«dlatt Super- 


Both 




Providing 




Inttrval 


Afrtcs 


visor AffMS 


visor AfrMS 


Afrtt 


Total 


the Data 




000 1 50-000 349 


9 


3 


6 


5 


23 


6 






39.13 


13.04 


26.09 


21.74 


100.00 




04 


000 350-000 649 


96 


26 


20 


45 


187 


38 




Row % 


51.34 


13.90 


10.70 


24.06 


100.00 




OS 


000 650-001 049 


354 


68 


87 


226 


735 


66 




Rnw % 
IVUW /o 


48 16 


9 95 


1 1.84 


30.75 


100 00 

1 W/.Vv/ 




0/> 


001 OSO— 001 S4Q 


398 


147 


159 


374 

.7 / *T 


1 001 


81 






39 77 


14 69 


15 18 


37 36 


100 00 




07 


001 S ^0—009 140 


483 


938 


108 


486 


1 405 


1 17 




tvow /o 


34 38 


16 04 


14 00 


34 SO 


100 00 




08 


007 1 50-009 849 


368 


188 


1 54 


539 


1 949 


194 




Row % 


99 63 


15 14 


12.40 


42.83 


100 00 




09 


009 850—003 649 


9 S3 


197 


146 


486 


1 019 


133 




Row % 


9S 00 


19 55 


14.43 


48 09 


100 00 




10 


003 650-004 549 


1 10 


63 


77 


960 


510 


1 10 




Row % 


21.57 


12.35 


15 10 


SO 08 


100 00 




11 


004 550-005 549 


87 


38 


56 


202 


383 


85 




Row % 


22.72 


9.92 


14.62 


59 74 


100 00 




12 


005.550-007.549 


144 


70 


89 


258 


561 


1 17 




Row % 


25.67 


12.48 


15.86 


45.99 


100 00 




13 


007.550-011.549 


102 


52 


54 


191 


399 


98 




Row% 


25.56 


13.03 


13.53 


47.87 


100.00 




14 


011.550--019.549 


40 


23 


31 


133 


227 


68 




Row% 


17.62 


10.13 


13.66 


58.59 


100.00 




15 


019.550-up 


16 


19 


13 52 


100 


39 






Row% 


16.00 


19.00 


13.00 


52.00 






Total 




2,390 


1,062 


1,083 


3,250 


7,785 


295 


Total 


Row% 


30.70 


13.64 


13.91 


41.75 


100.00 





Nott. —The agreement columns arc mutually exclusive. Example, in interval 05 the other supervisors actually agreed 
with incumbents that 68+226 tasks were performed, for a total of 294, or 40.00 percent. 
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Table J. Supervisor-Incumbent Agreement on Tasks Performed, 
Inventory Management, 645X0 







00 


10 


01 


1 1 




incumtMntt 






N«lth«r 


Oth«r SuP«r- 


Imrmdiat* SuPtr- 


•oth 












vltor Afir*#t 




airn 


TotsI 




03 


000.150-000.349 


2 


3 


1 


5 


11 


3 




Row% 


18.18 


27.27 


9.09 


45.45 


100.00 




04 


000.350-000.649 


55 


11 


15 


33 


114 


26 




Row% 


48.25 


9.65 


13.16 


28.95 


100.00 




05 


000.650-001.049 


212 


47 


67 


199 


525 


43 




Row% 


40.38 


8.95 


12.76 


37.90 


100.00 




06 


001.050-001.549 


197 


79 


108 


260 


644 


53 




Row% 


30.59 


12.27 


16.77 


40.37 


100.00 




07 


001.550-002.149 


296 


141 


129 


306 


872 


77 




Row% 


3354 


16.17 


14.79 


35.09 


100.00 




08 


002.150-002.849 


225 


122 


114 


349 


810 


79 




Row% 


27.78 


15.06 


14.07 


43.09 


100.00 




09 


002.850-003.649 


180 


65 


99 


257 


601 


86 




Row% 


29.95 


10.82 


16.47 


42.76 


100.00 




10 


003.650-004.549 


71 


42 


42 


101 


256 


60 




Row% 


27.73 


16.41 


16.41 


39.45 


100.00 




11 


004.550-005.549 


60 


22 


31 


^34 


247 


56 




Row% 


24.29 


8.91 


12.55 


54.25 


100.00 




12 


005.550-007.549 


91 


46 


54 


136 


327 


72 




Row% 


27.83 


14.07 


16.51 


41.59 


100.00 






\AJ / .«) JU— U 1 i *OHy 


OO 


JO 


jZ 


1 


ZjO 


OJ 




Row% 


26.56 


14.06 


12.50 


46.88 


100.00 




14 


011.550-019.549 


34 


15 


21 
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Total Incumbents 
Total Supervisors 
Total Tasks Reported 
Total Tasks Identified 



1- 

04 

11 114 

3 26 



09 



10 



11 



T2 



191 
382 
4,892 
3,390 



247 327 256 152 77 
56 72 65 46 30 



Time Spent 
Number of Tasks 
Contributors 



1^ 
03 



05 



06 



525 
43 



07 

644 872 
53 77 



08 



810 601 256 
79 86 60 



Figure L Percent agreement of supervisors with inventory management, 645X0, 
personnel at all levels on tasks performed. 
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/ / Immediate Supervisor 



Total Incumbents 104 

Total Supervisors 208 

Total Tasks Reported 2,893 

Total Tasks Identified 2,005 



-| ! 1 1 1 1 1 1 1 — 

Time Spent 03 04 05 06 07 08 09 10 11 12 
Number of Tasks 12 73 210 357 533 432 411 254 136 234 143 75 23 
Contributors 3 12 23 28 40 45 47 50 29 43 33 22 9 



Figure 2. Percent agreement of supervisors wfith materiel facilities, 647X0, 
personnel at ail levels on tasks performed. 
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The inventory management data have been broken out into lower and upper skill levels in Figures 3 
and 4. What appeared to be a regular slope in Figure 1 is seen to have been mainly contributed by the lower 
skill level observations. Supervisors in both the immediate and other status fafled to tally the tasks of lower 
level airmen on which small amounts of time were claimed, if one takes 50% tallying as a standard. 



I 
I 
I 

I Total Incumbents 118 

Total Supervisors 236 
I Total Tasks Reported 2,060 

j Total Tasks Identified 1 ,268 




i I 1 1 \ 1 1 1 1 1 1 1 1 — 

TimeSpent 03 04 05 06 07 08 09 10 11 12 13 14 15 

Number of T?sks 7 24 93 187 274 342 253 133 1 33 229 187 117 73 

Contributors 2 7 15 19 31 36 46 30 33 48 45 34 88 



Figures. Percent agreement of supervisors with lower level inventory management 
personnel, 64530 and 64550, on tasks performed. 
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Other Supervisor 



Total Incumbents 
Total Supervisor 
Total Tasks Reported 
Total Tasks Identified 



73 
146 
2,832 
2J22 



i J 1 I I I 1 1 1 1 1 1 1 

Time Spent 03 04 05 06 07 08 09 10 11 12 13 14 15 

Number of Tasks 4 90 429 457 593 468 348 123 114 98 69 35 4 

Contributors 1 19 28 34 46 43 40 30 23 24 20 12 2 

Figure 4. Percent agreement of supervisors with upper level inventory management 
personnel, 64570 and 64590, on tasks performed. 



The materiel facilities data of Figure 2 have been broken out into lower and upper skill levels in 
Figures 5 and 6. Although the number of higher level materiel facilities personnel is too small to provide 
stable data, the tally level tends to exceed 50%. In general, the Materiel Facilities supervisors appear to tally 
on a greater percentage of their subordinates' tasks than do the supervisors in Inventory Management. 

The data of Table 3 and tables not shown, from which graphs we're drawn to make Figures 1 through 
6, have been summed to form Table 4, and to show supervisor-incumbent agreement on tasks performed 
without using time spent values. Collapsed in this fashion, the data reveal that the poorest 
supervisor-incumbent agreement occurred for the lower level Inventory Management sample. 
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Figure 5. Percent agreement of supervisors with lower level materiet facilities 
personnel, 64730 and 64750, on tasks performed. 
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Figure 6. Percent agreement of supervisors with upper level materiel facilities 
personnel, 64770 and 64790, on tasks performed. 



Table 5 presents data on the agreement of supervisors with subordinates on tasks indicated as not 
performed. This sliould not be thouglit of as merely the zero point on the time spend scale of Table 3. 
Although Table 5 is arranged like Table 4, the data are qualitatively different from data on tasks performed. 
A glance at the number of tasks not performed should be sufficient to see that a high percentage of 
agreement on nonperfomiance could occur, despite inaccuracy on the part of the rater. To illustrate this, 
allow a single incumbent to represent all incumbents. If he reported performance on 20 of the 281 tasks in 
the inventory, and if a supervisor had rated him on the performance of 10 tasks, all of which were wrong, 
the data would show that the supervisor was right in 231 of the 261 possibilities. The result would be 89% 
agreement. The illustration given is an extreme one, applying only to airmen with very few tasks. The more 
tasks performed, the less chance for agreement. 

The four types of agreement (00, 10, 01, 11) remain the same in Table 5 as in the preceding tables, 
but logical interpretations of these tallies may be different. The 00 column, or both supervisors rated 
incumbents on the pcrfomiance of tasks that were not claimed, is of interest. As indicated above, a single 
supervisor could "agree" with an incumbent on the nonperformance of tasks as a probability function of 
the number of tasks in the inventory ; however, the combined disagreement of two supervisors is a less likely 
occurrence. Table 5 shows this type of disagreement to be from 5% to 6% for the lower level airmen, and 
from 7% to 10% for upper level incumbents. The primary observation one can make from Table 5 is lack of 
superiority of one level of supervisor over another, which conforms to the findings made on tasks that were 
claimed by the incumbents. 



IV. CONCLUSIONS AND DISCUSSION 

This study shows no systematic or observable difference between inunediate supervisors and other 
supervisors in agreement with their subordinates. That is, immediate and other supervisors rated 
performance on the same percentage of tasks on which their subordinates had reported time spent. 

When cases by skill level were pooled the resulting figures indicated that the more time a subordinate 
spent on a task the more likely was his supervisor to rate him on it. However, when the data were broken 
down into skill levels, greater agreement was found for upper skill level incumbent tasks when the data 
involved 20 or more incumbents. A low level incumbent had to report more time spent on a task to achieve 
the sanie level of supervisor agreement. The analyses offered no way of telling whether this was due to 
supcrvfsor ignorance about lower level airmen, or to a tendency of lower level airmen to claim tasks which 
supervisors rejected. 

The major finding of the study 4s quite clear - one docs not have to limit himself to immediate 
supervisors in order to find performance raters who know what a subordinate is doing. However, it would 
be a mistake to generalize from this study alone. For example, the data appeared to follow a consistent 
slope when 20 or more incumbent inventories contributed a task at a given time spent level. This is 
suggestive of procedures for specifying the amount of data needed to identify a task reliably; but 20 is not a 
magic number and the data from this study may not be typical of Air Force airman jobs in general. 

The major findings challenge one's credulity. It just doesn't seem logical that anyone could know as 
much about an airman's job as his immediate supervisor. Before dismisang this belief as disproven it may be 
well to examine the underiying assumptions. 

First, there is the designation "immediate supervisor." When a base personnel office is asked to 
transmit rating materials to units, as was done in this study, it is likely that the "inmiediate supervisor" 
selected would be the person responable for the airman's official performance report. This supervisor may 
not necessarily be the most proximate individual. It is suggested that a check list or questionnaire approach 
could yield a score that would identify the closest supervisor, uiio might not necessarily carry the title 
"immediate." 

Second, there was the fairiy high tally percentage for supervisors of upper level airmen. Why should 
this occur, when the higher level NCOs had far more tasks? The instructions made it clear that so much 
time was being asked of the rater that no supervisor could be expected to rate more than one airman. 
Although some supervisors rated two airmen, it is probable that the units made a special effort to select 
knowledgeable supervisors. In other words, the other supervisors chosen were really as familiar with 
subordinate jobs as the supervisors who were "immediate." 
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In essence, this study indicates that many supervisors are knowledgeable about their subordinates* 
jobs and that the designation **imniediate supervisor" may not always be what it seems to be. 

It was found that in the supply area it is possible to obtain two supervisors who have about equal 
knowledge on which to base performance ratings of airmen. The finding had direct impact upon analyses 
concerned with developing performance criteria. If it should turn out that rater equality was limited to 
DAFSCs 645X0 and 647X0, llie finding would have very little general value. However, if later studies 
sliould demonstrate that the finding is characteristic of many ladders in the Air Force, the problem of 
collecting performance data would be lessened. Better data migltt be collected by identifying the nature and 
frequency of a supervisor's contacts witli his ratee than by merely looking for an "immediate** supervisor. 
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